Repair Pipelining for Erasure-coded Storage: Algorithms and Evaluation

We propose repair pipelining , a technique that speeds up the performance in general erasure-coded storage. By carefully scheduling of failed data small-size units across storage nodes pipelined manner, reduces single-block time to approximately same as normal read for single block homogeneous environments. further design different extensions algorithms heterogeneous environments and multi-block operations. implement prototype, called ECPipe integrate it middleware system into two versions Hadoop Distributed File System (HDFS) (namely, HDFS-RAID HDFS-3) well Quantcast System. Experiments on local testbed Amazon EC2 show significantly improves degraded reads full-node recovery over existing techniques.